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ABSTRACT 


This paper presents some preliminary results of the application of multi- 
dimensional scaling methodology in human factors engineering. The non-ortho- 
gonality of internally perceived task variables is exhibited for first and 
second order plants with both dependent and independent task variables. 
Directions of operator preference are shown for actual performance, pilot 
opinion rating, and subjective measures of fatigue, adaptability and system 
recognition. Improvement of performance in second order systems is, in 
addition, exhibited by the use of bang-bang feedback information. New 
dissimilarity measures for system comparison are suggested in order to 
account for human operator rotations and subjective sense of time. 


1 . INTRODUCTION 


In comparing the objective performance of a human operator (H.O) 
with his/her subjective evaluations it was found helpful in reference Q] to 
use the methodology provided by multi-dimensional scaling (MDS). According 
to this methodology, first proposed in detail by Torgerson in 1952 [23 in 
the area of mathematical psychology, unidimensional pilot opinion rating 
scales (POR) are only to be thought of as the vecotrs in a multi-dimensional 
space which represents the "perceptual model" the H.O. has "internally 
constructed" on the nature and performance of his/her own task. 

This internal task space (ITS) is conceptually different from 
the ones arrived at by deviate internal models of the H.O., as for example 
done in reference C3], in that the MDS formulation does not rely on the LQ6 
models of the H.O. and is in essence completely model -free. What this ITS 
really depends on is the metric assumed to apply in constructing it from 
comparison measurements provided subjectively by the H.O. and interpreted 
mathematically via the MDS formulation. These comparisons fix the 
distances between the compared objects within the multi -dimensional space 
and correlate with the unidimensional subjective (SE) =*nd objective 
evaluations (OE) of various system parameters via the use of a vector-fit- 
ting algorithm. 
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A complete analysis of the mathematical model employed including 
the real-time computational aspects involved in updating the ITS can be found 
in reference C4a]. The dimensionality of the matrix POR depends on the 
total number of tasks, compared, and more precisely 

r" n > N & log^ (n^^g^ + 1) = n 

where n the dimensionality of the ITS 

N the number ■■ task variables .. 

in. the # of selected values for the i - variable, and 

n’ the geomocric average of n^'s. 

This logarithmic complexity of MDS experiments makes it 
imperative to: 

i. partition the original set of tasks into partially overlapping smaller 
groups which will still retain the property of clustering them back 
together "under the same roof" of a single ITS; 

ii. design separate experiments with the smaller groups employing a small 
number of selected values for each vari ble; 

iii. reduce mathematically v’a a statistical hyperplane fit the total 
number of dimensions towards .he number of actual task variables 
(if these are knov/n beforehand and the experiments well controlled). 

The prescribed technique has been successfully utilized by 
Siapkara C4b] in performing a series of three sets of experiments with three 
subjects and two task variables. The resulting two-dimensional ITS, though 
certainly of limited validity, does effectively portray ma"y of the 
associated MDS issues. Section 2 of this paper describes the experiments 
and section 3 deals with the experimental results. All accompanying figures 
can be found at the end of the paper. 


2. THE SCOPE OF THE EXPERIMENT 


Figure 1 shows in a self-explanatory way the experimental set-up 
used. The signal source is a smoothed slowly-varying output from a random 
number generator (RNG), so that the configuration of the systems to be 
controlled [Figure 2.1 corresponds to both a first and second-order system.^ 
Which of the systems is actually controlled by the H.O. depends on which of’ 
the two displays is accessible for consultation, and not on any difference 
in plant dynamics. In order to keep a small number of variables at hand, the 
parameters of both systems do not vary independently [Figure 31; they are 
all instead in terms of a common parameter A. 

The second task variable K comes from the input. An effort 
was made to use a random-number generator which experiences minimal 
statistical variations over time and different starting values, so that only 
its mean strength (amplitude) will count. The generation method was' based 
on a combination of techniques proposed by references [51 and [61. 
Consistency of the smoothed random input was checked for two completely 
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different sets of task variables and measurement policies. [Figure 43. It 
also checked comparatively well with the sinusoid method used in reference 
C7], with its statistics attaining an asymptotic stability much sooner, 
though the variations of the sinusoidal RNG are less frequent. 

The use of fixed analog displays did not allow for the 
appropriate scaling of feedback indication, as would be the case with a 
flexible CRT screen. So, instead of performing the image range calibrations 
as outlined by equation (3) of reference Q], it was necessary to keep the 
number of display overshoots as an objective performance index in addition 
to tracking error scores. The sprin-j characteristics of the vertical level 
knob used in the experiment was what Dommasch C8D would call a "bungee" 

(or down spring) control element, with a built-in center-wards pull which 
requires a constant off-center push in part of the subject in the case of 
proper control-force static stability. Transformation of the results to a 
situation which uses different control -element characteristics can be done 
via Rothbauer C93 [Figure 53. Performance curves of these types of 
controlled elements is given in reference [103 [Figure 63. 

Finally, provision was made in the experimental set-up to 
include a white-red light depending upon whether the tracking error was 
positive or negative. Experiments based solely on this type of feedback 
information will here-in be referred to as sign experiments. Justification 
for the inclusion of such an experiment is based on previously acquired 
insight in the field of experimental psychology. Johnson [113 mentions a 
28-person 1968 experiment where multi -dimensional judgements correlated well 
with uni-dimensicnal equivalents when simple combinatory transformations 
of the various variables were performed. It was found that 42% judged on 
linear scales, 10.5% on quadratic, 45.5% on signed cues, and the rest in 
other configural modes. The signed cues were indicators with either +1 
or -1 values, and they contained the sign information of the judgement 
only. For example, in the MDS context of paired comparisons the subjects 
would actually judge as if the stimuli were near the vicinity of 
just-discriminable differences: instead of rendering refined estimates of 
similarity, they would rather ask themselves some more foundamental 
questions: "Are the two situations cxitpeured different enough so as to bother 

giving out discrete and even more so continous estimates on 
scales beyond binary? And, if I admit there is a perceptible 
difference between them, will I be able later on on subsequent 
pair ccnparisons to maintain seme credible consistency of how I 
rate these minute differences? Or, is it possible that I am 
going to develop the tendency of accentuating the dissimilarity 
scale near the similarity end of it, and oenpfess thus ray judge- 
ments on the truly dissimilar cases?" 


3 . EXPERIMENTAL RESULTS 


Nine first and nine second order systems in all were evaluated. 
They were gotten by combining three values from each of the two variables 
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as shown in Figure 7. The same figure shows also the uncertainties and just- 
discriminable differences related with each of these task variables. Various 
types of subjective judgements rated on a ' 10 scale were collected in the 
experiments for these systems. The verbal characterization of these scales 
was only fixed at the ends, and is presented in Figure 8. For each category 
of systems there were essentially three kinds of runs performed; combined 
familiarization-evaluation runs, dissimilarity runs, and finally identifica- 
tion runs. For each run, in addition to the particular subjective judgement 
aimed at, objective performance records were kept, as well as subjective judge- 
ments on the level of tatigue experienced with the experiment and the level of 
difficulty (effort) pronouncing the subjective evaluations themselves. 

Familiarization - evaluation runs 
lasted 2 mins for each system. In the first 20 secs of familiarization SE on 
the success of familiarization was taken verbally around every 5 secs 
[Figure 9, located under Figure 63. For 1-st order systems as Xt so in 
general did the difficulty for familiarization (ex. VIII), but in no circum- 
stance did the degree of familiarization decrease as time progressed. On the 
contrary 2-nd order systems with K4- experienced such a drop (ex. VII). 

The next 10 secs were considered a break between the familiarization portion 
of the run and the evaluation portion. During this interval a combined SE was 
provoked that indicated the efficacy of this relaxation period and the degree 

of i'*nfort the subject experienced [Figure 10(a) 3. The direction of 

relaxation increase is definitely different for 1-st and 2-nd order systems. 
The middle 60 secs of the runs are devoted to system evaluation. The subject 
with uninterrupted concentration on the task was previously instructed to 
perform his best. At tne end of this period he/she gives an overall POR on 
the task [Figure 10(b) ---3 , while OE are recorded for future compari- 
son [Figure 10(b) 3. While OE increases with K4-,X4- for both 1-st and 

2-nd order systems, SE's in general do not. Separated areas indicate entries 
which did not comform with the general direction of the property vectors and 
differ by more than one point in the psychological scale from what would have 
been considered as a value in acceptable deviation from the rule. 

Along with the POR, the subject indicates his/her own mental and 
dexterity fatigue status, and the effort expended by him/her in rendering these 
SE's [Figure 10(c) 3. A general kind of agreement can be seen for both SE 
measures used, while the scales utilized by che subject differ by one and two 
points, the subject being more harsh on rating the fatigue factor. The last 
30 secs of these runs are used for deadaptation purposes till the start of 
the subsequent run. Figure 10a (vectors in segmented line) shows the degree 
of comfort felt by the H.O. It can be seen that inter-run deadaptation does 
not relate to intra-run relaxation, though both are comfort accomodating. This 
is so because relaxation on the same task is viewed by the H.O. simply as a 
means to reduce his/her fatigue, whilst deadaptation seems to depend more 
upon the anxiety of what comes next. 

Dissimilarity runs lasted 70 secs for each 
pair of systems compared. Each member of the pair was controlled for 30 secs 
at the end of which OE was recorded. Figure 11 shows the variations in 
performance relative to the OE of evaluation runs as a standard of reference 
for both types of systems. 10 secs in between the individual presentations 
served as a relaxation during which the degree of case for remembering the 
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behaviour of the system presented first was recorded [Figure 10(d)]. The 
incomplete nature of the memorization space results from the factorial design 
of the dissimilarity runs which forms a minimum number of combinations to be 
compared. Despite the lack of additional information the essential character 
of the memorization vectors is evident, and suggests that the impression ot 
remembering a system remains in direction the same for 1-st and 2-nd ordar 
systems. However, this does not necessarily mean that actual ability for such 
memorization is so. This point is discussed in more detail later on. At the 
end of the dissimilarity runs the H.O. judges the similarity of the systems 
compared, as well as his/her own degree of comfort in pronouncing this judge- 
ment. An off-line procedure for computing two-dimensional ITSs is invoked 
at the end of all the dissimilarity experiments. 

The three sets of experiments involved, tasks v -vi^-ix , I -ii - 
-III -IV -VII -IX, and l.-il^-lll.-iv. . After finding the' task 

vectors based on these partitioned experiments, the ITS's are brought together 
by modifying the point dispersions so that the task vectors coicide [Figure 
13(a)]. The rest of the systems viii^ and V 2 -IX 2 are then placed within 
this combined ITS by a straight-forward two-dimensional interpolation proce- 
dure. The same is done with objective performance and subjective evaluation 
vectors for both 1-st and 2-nd order systems [Figures 13(b)+(c)]. 


4. DISCUSSION 


Many could be the implications of these configurations, if it was 
not for the limited evidence for these internal space construction^ ! What is 
for sure is that the evidence collected on the few subjects of the experiments 
exhibits on the average tendencies which were sort of anticipated, and which 
motivated this study in the first place, anyway. These tendencies are: 

i. the task vectors are not in general perceived independently, 

ii. objective and subjective ratings of manual tracking tacks do not 
necessarily coincide. 

The large inconsistencies in comparison distances (offsets that are 30fb off 
from the Euclidean point of view), and the quite considerable variations in 
performance (uncertainties of the order of 36.4% ) raised a number of 
questions on the validity/utility of MDS in the multi -dimensional assess- 
ment of POR's. 

This motivated a third stage in the experiments beyond familiari- 
zation-evaluation and dissimilarity runs. More specifically, identi- 
fication runs were specially conducted to test the hypothesis 
inherent in these experiments, that the H.O. could identify successfully 
the differentiating character of the systems he/she is confronted with. Figure 
lOd (vectors in segmented line) shows the directions of maximum increase 
in actual memorization, assuming that identifiabil ity is stricty speaking a 
measure of memorization. In certain cases an almost complete lack of ability 
to identify a specific task is evident, as for example: ill ,vii ,iv, and rii . 
[Figure 12]. The short duration of the tasks can be cited ^ as "^a ‘main 
cause, because identifiabil ity is a cumulative property which combines together 
the temporal elements of a task; and if the task has not fully developed its 





essential idiosyncracies this plays a negative role on the H.O. perceiving 
its global nature. Figure 14 shows, for example, the immense variability in 
the character of an almost marginal 2-nd order task in 10 secs intervals. 
This particular task reveals its true collapsing character after 50 secs, and 
certainly not within the first 30 secs from its activation. On the other 
hand, difficulties in memorization are unrelated to the factorial design of the 
similarity experiments. Figure 15 clearly shows that fatigue accumulated on 
entire groups of comiparisons (based on the same first member of the pairs) 
does not seem to relate directly to the difficulty associated with identifying 
those systems. 

Finally, the improved performance shc.vn with sign rather than 
complete information on tracking error CFigure 16ia)l motivated a correla- 
tion between the various systems and the number of bang-bang pieces of feed- 
back information. Figure 16(b) suggests that the implicit strategy used by 
the H.O. in optimizing manual tracking performance is to try to reach a uni- 
form level of acceptance in the number of tracking error crossovers. 


5 . FUTURE DIRECTIONS 


Despite some of the disheartening aspects of applying MDS on POR 
pronouncement, this has much more to do with H.O. inconsistencies than with 
a methodological difficulty and/or inability inherent in MDS itself. To the 
contrary, reference r.4c] suggests that an MDS approach to the problems of 
mental workload in a multi-task environment and of multi -operator judgement and 
control (collective task-attending) would simplify their study by avoiding 
the use of the law for comparative judgement C12] and of group probability 
partitioning C13] , respecti'/ely. 

Relating individual ITS's of various tasks or various operators, 
— ^firstly between themselves and secondly with the mo*“e complex ITS's result- 
ing from multi-task or multi-operator situations — , would in the opinion of the 
authors provide us with usuful POR matrix transformations; for example: 

i. between groups of people with different levels of aptitude in performing 
manual tracking tasks, 

ii. for increasing the reliability of the operation by providing feedback 
information to the H.O. about disc»*epancies in his/her ITS between 
objective performance and its subjective evaluation, 

iii. for the design of flexible control/ splay configurations which will 
automatically adapt their dynamics a«-cording to the ITS peculiarities 
of t^e H.O. involved in the operation so as to improve performance in 
a waj transparent to the H 0., etc. 

Finally, it has to be noted that Euclidean or even Minkowski spaces 
provide metrics for ITS that are not suitable to express the contribution of 
terms that corresocnd to situations where H.O. space rotations and his/her 
subjective sense of time (the time thought of as having been el lapsed in the 
controlling action) play an important role. This is so because the base 
vectors representing a space rotation combine in a multiplication grrup C14] 
and those representing time belong to the spinor class C153. This can be 
illustrated as follows: 
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Consider y = x,§^ + + + x 

I ^ " 1/p 

and |y l]=CSjxJ + s^x^ + + s'^xP] (1) 

where only superscript p is a. power. Then for variables x. that constitute 

i. an inner product group, s’= +1 

ii. elements of a space rotation. 




© 



k 


P 


where k^,k 2 , kp in circular order 

iii. a designant for the H.O.. sense of ellapsed time (when experiments 
seem to depend on it), s - -1. 


It can be seen that (1) applies only in the first and 
the metric corresponding to the second case is given by 

||y|= X|J 

12 p 


following reference C 143, the summation convention and 

|ylh CiCSi© ©£3 


third cases, whereas 
the more general form 

( 2 ) 


the definition 


The double-circle operator might be any legitimate operator, such as an inner 
or outer product, an integration of base functions, or even a convolution 
integral in the case of cascaded moving vectors, or a meaningful mixture of 
the above. 


Whereas solving MDS with metrics of the form (1) seems to be 
a trivial extension of the case where s^=l ^ i, this is certainly not the 
case with the much more general form (2), where coupled terms do in general 
appear. 
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FIGURE 9 
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FIGURE 5 
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FIGURES 10 

u) Intra- and Inter-Run Relaxation Vectors 

(relaxation deadaptat Ion- -) 

(b) Performance And Judgement Vectors 

(objective performance , 

subjecti/e e'^aluation----) 

(c) Subjective Stress Status Vectors 

(fatigue , effort ) 

(rt) Memorization Vectors 

(aabjective , ob j ec t i ve — - --) 
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FIGURES 13 

Property Vectors 

^ ^ (in the Dissimilarity ITS ) 

L subscripts signify order of the system; x signify predicted ii'cations 
of the remaining systems; circled numbers signify 2-nd order systems 3 
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(tasks are rank-ordered according to identi liability) 




